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Concerned about the expense of teacher turnover, its disruption to schools and students, and its potential 
effect on students’ access to effective teachers, the School District of Philadelphia partnered with the 
Regional Educational Laboratory Mid-Atlantic to better understand students’ access to effective teachers 
and the factors related to teacher turnover. This analysis of differences in teacher effectiveness between 
and within schools in the district found that teachers of economically disadvantaged, Black, and Hispanic 


students had lower evaluation scores than teachers of non—economically disadvantaged and White 
students but similar value-added scores (a measure of teacher effectiveness based on student academic 
growth). The study also found that each year from 2010/11 through 2016/17, an average of 25 percent of 
the district’s teachers left their school and 8 percent left the district. During the first five years of teaching, 
77 percent of teachers left their school and 45 percent left the district. Turnover rates were highest for 
teachers who taught middle school grades, teachers who missed more than 10 days of school a year, 
teachers who identified as Black, teachers who had previously changed schools, and teachers who had 
low evaluation ratings. Teacher turnover was higher in schools where teachers had a less positive view 
of the school climate. School climate mattered more for teachers with higher evaluation ratings than for 
teachers with lower evaluation ratings. 


Public schools across the United States lose about 16 percent of their teacher workforce annually, with 8 percent 
of teachers leaving the profession (Carver-Thomas & Darling-Hammond, 2017). Teacher turnover—the percent- 
age of teachers leaving their school or their district—is higher in most urban districts than the national average, 
including in the School District of Philadelphia (SDP; Carver-Thomas & Darling-Hammond, 2017; Steinberg et al., 
2018). The most recent study of turnover in SDP found that from 2009/10 through 2015/16, 27 percent of teachers 
left their school and 15 percent left the district each year on average (Steinberg et al., 2018). That study examined 
turnover by teacher experience and subject taught, school demographic characteristics, and the percentage of 
students scoring proficient on state assessments in the school, but it did not examine turnover by other import- 
ant teacher and school characteristics, such as teacher effectiveness, teacher attendance, teacher race/ethnicity, 
and school climate. 


Nationally, turnover is higher in districts where schools have more lower-performing students, economically dis- 
advantaged students, students of color, or students requiring special education services (Boyd et al., 2005; Carv- 
er-Thomas & Darling-Hammond, 2017; Goldhaber et al., 2011; Sullivan et al., 2017). In addition, new or younger 
teachers, math and science teachers, special education teachers, and teachers without standard certifications are 
more likely to leave their school or the profession (Nguyen et al., 2019). 


Because teacher turnover can be costly to districts and impair student learning, 
it is valuable for districts to understand how teacher and school characteristics 
contribute to turnover. For example, districts might derive a net savings from 
teacher turnover if experienced teachers are replaced by newer teachers who 
earn lower salaries. But if a teacher leaves after five years of teaching in a dis- 
trict, the costs of training a new teacher and the lost productivity associated 
with hiring a replacement teacher may outweigh any salary savings (Milanowski 
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& Odden, 2007). Estimates of the net cost of turnover range from as low as $6,800 to as high as $33,400 per 
vacancy (Milanowski & Odden, 2007). The overall costs of teacher turnover can be substantial in a district as large 
as SDP, which hires 1,000 teachers a year to fill vacant positions (Sasko, 2017). In addition to the financial costs, 
the disruption to the classroom and to the school community from high turnover may result in lower student 
achievement (Henry & Redding, 2018; Ronfeldt et al., 2013). 


SDP is also concerned that teacher turnover could exacerbate inequities in its schools. A study of turnover in 
SDP found that teachers who changed schools left schools that had more students of color, more economically 
disadvantaged students, and lower student achievement than the school they moved to (Steinberg et al., 2018). 
Turnover can widen gaps in school quality if vacancies are filled by teachers who are less effective than those who 
left and if teachers who are more effective move to higher-performing schools (Boyd et al., 2008; Feng & Sass, 
2017; Goldhaber et al., 2011; Hanushek et al., 2016; Krieg, 2006; Ronfeldt et al., 2013). 


However, some research indicates that the effectiveness of teachers of economically disadvantaged students 
differs only slightly from that of other teachers and that the movement of teachers between schools is unlikely to 
contribute meaningfully to differences in access to effective teachers, in part because less effective teachers tend 
to leave their district entirely rather than move to another school in the district (Chetty et al., 2014; Hanushek & 
Rivkin, 2010; Isenberg et al., 2016; Mansfield, 2015; Steele et al., 2015). Yet other research finds substantial vari- 
ation across districts in equitable access to effective teachers, with some districts having large gaps in teacher 
effectiveness according to student income or race/ethnicity and other districts having small or no gaps (Gold- 
haber et al., 2018). 


Studies have not examined the extent to which teacher effectiveness varies across schools within SDP or whether 
SDP students from different income or racial/ethnic groups are taught by teachers with different levels of effec- 
tiveness. In addition, studies of turnover in SDP have not examined whether teachers who change schools are 
more or less effective than average, information that could help SDP assess whether teacher movement is con- 
tributing to gaps in access to effective teachers. 


Concerned about the expense of teacher turnover, its disruption to schools and students, and its potential effect 
on students’ access to effective teachers, SDP partnered with the Regional Educational Laboratory Mid-Atlantic to 
better understand students’ access to effective teachers and the factors related to teacher turnover. The district 
is already working to reduce teacher turnover by supporting new teachers, targeting retention strategies accord- 
ing to teacher needs, and improving school climate. For example, SDP began a series of professional development 
offerings for new teachers, along with new teacher induction and additional supports during a teacher’s first 
year in the district. To encourage experienced teachers to stay, SDP began providing professional development 
opportunities customized to teacher and school needs. SDP also uses exit surveys and other data on turnover 
trends to target retention strategies, and it plans to expand its work with principals to improve school climate and 
school-level retention. 


This study provides information that can inform these initiatives. Findings on the distribution of teacher effective- 
ness across the district, and on whether effectiveness is related to turnover, can improve SDP’s understanding of 
the role of turnover in the equitable distribution of effective teachers. Findings on the relationship between other 
teacher characteristics—such as teaching experience and grade level taught—and turnover could inform reten- 
tion strategies. Findings on the relationship between school characteristics—such as school climate and student 
achievement—and turnover could inform SDP’s work with principals on improving school-level retention. 
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Research questions 


This study addressed the following research questions on the distribution, effectiveness, and turnover of teachers 
in SDP for school years 2010/11 through 2016/17: 


1. How are teachers with different evaluation ratings distributed across schools in the district? Does student 
access to effective teachers vary between and within schools? 


2. What proportions of teachers leave their school and leave the district each year, both during the school year 
and between school years? 


3. How does teaching experience relate to whether teachers leave their school or leave the district? 
a. What other teacher and school characteristics are related to teachers leaving their school or leaving the 
district? 
b. Does the relationship between teacher and school characteristics and turnover differ between effective 
teachers and ineffective teachers? 


Research question 1 examines whether effective teachers are evenly distributed across district schools. It also 
examines whether economically disadvantaged, Black, and Hispanic students have equal access to effective 
teachers—whether these students and non-economically disadvantaged students and White students have 
teachers with similar effectiveness scores and value-added scores. 


Research question 2 examines the average teacher turnover in the district from 2010/11 through 2016/17 and 
whether turnover was especially high or low in any year. 


Research question 3 examines the turnover rate during each year of a teacher’s career and the probability that 
teachers will leave as their career progresses, which indicates the total amount of turnover a school or the district 
is likely to experience over time. Research questions 3a and 3b examine whether teacher and school characteris- 
tics are related to turnover after multiple teacher and school characteristics are accounted for. 


Definitions of key terms used in this report are in box 1. The study’s data sources, sample, and methods, as well as 
limitations of this study, are described in box 2 and detailed in appendix A. 


Box 1. Key terms 


Early career teachers. Teachers in their first four years of teaching in the School District of Philadelphia (SDP). 


Economically disadvantaged students. Students eligible for free meals through income-based programs such as the Supple- 
mental Nutrition Assistance Program or Temporary Assistance for Needy Families. 


Effective and ineffective teachers. This report refers to teachers whose teacher effectiveness score or Pennsylvania Value- 
Added Assessment System score (value-added score) would give them a rating of proficient or distinguished as effective teachers 
and to teachers whose effectiveness score or value-added score would give them a rating of failing or needs improvement as 
ineffective teachers. 


Effective teaching gap. A districtwide measure of the difference in average effectiveness scores and value-added scores of teach- 
ers of economically disadvantaged, Black, and Hispanic students and the scores for teachers of non—economically disadvantaged 
and White students. 


Elementary school grades. Kindergarten-grade 5. 
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Frequently absent. A teacher attendance rate between 75 percent and 94 percent. Teachers with an attendance rate below 
75 percent are excluded from this definition because they might have extenuating circumstances leading to more absences, such 
as medical issues or maternity leave. 


High school grades. Grades 9-12. 


Intraclass correlation coefficient. A measure of how similar teachers are within the same school and how similar each school’s 
average teacher is to the average teacher at other schools. Intraclass correlation coefficients report the percentage of the total 
range of teacher effectiveness scores or Pennsylvania Value-Added Assessment System scores that can be explained by differenc- 
es between each school’s average teacher. 


Middle school grades. Grades 6-8. 


Pennsylvania Value-Added Assessment System (PVAAS) score. In the SDP educator evaluation system, the PVAAS score 
(value-added score) is a measure of a teacher’s effectiveness based on his or her students’ academic growth on a state math or 
English language arts assessment (School District of Philadelphia, 2017). 


Regular attendance. A teacher attendance rate of 95 percent or higher. 


Teacher effectiveness score. A summary measure used by SDP to rate a teacher’s overall effectiveness on a continuous scale 
from O (failing) to 3 (distinguished). Effectiveness scores correspond to the evaluation ratings as follows: O-0.49 is a rating of 
failing, 0.5-1.49 is a rating of needs improvement, 1.5—2.49 is a rating of proficient, and 2.53 is a rating of distinguished. Effective- 
ness scores incorporate information on teacher practices and student outcomes. Teacher observation scores, student learning 
objectives, PVAAS scores, and a school performance profile score are included in the effectiveness score (School District of Phila- 
delphia, 2017). 


Teacher evaluation rating. A four-category summary rating of teachers based on their teacher effectiveness score in the educa- 
tor evaluation system. A teacher may be rated failing, needs improvement, proficient, or distinguished. 


Teacher experience. Referred to as “years of teaching” in this report and measured from a teacher’s hire date in the district to 
exit date (if applicable). This could include time in a nonteaching position if a teacher was employed in a nonteaching position in 
the district before September 2010. 


Teacher survey. A districtwide survey that captures teacher responses to questions related to school improvement, including 
school climate, instruction, leadership, professional capacity, and parent/guardian community ties. The districtwide survey also 
includes surveys of students, principals, and parents or guardians; however, this study used only the teacher survey responses. 


Turnover rate. Percentage of all teachers who change schools within the district, leave a teaching position for a nonteaching 
position within the district, or leave the district. 


Veteran teachers. Teachers with five or more years of experience in the district. 


Box 2. Data sources, sample, and methods 


Data sources. The School District of Philadelphia (SDP) provided data for all research questions. (See table Al in appendix A for 
a summary of the data sources, variables, and years of data used in the study.) SDP provided data on teacher and school char- 
acteristics from 2010/11 through 2016/17, with several exceptions. For data on teacher effectiveness, Pennsylvania Value-Added 
Assessment System (PVAAS) scores were available only for 2014/15 through 2016/17, whereas teacher effectiveness scores were 
available for 2013/14 through 2016/17. For data on school characteristics the district provided teacher survey data for 2014/15 
through 2016/17 and data on the percentage of economically disadvantaged students in each school for 2012/13 through 2016/17. 


Sample. All analyses included staff with a job title that indicated that they were a K-12 teacher and excluded staff who worked 
in district administrative offices or preschools or who had a nonteaching role in a school. Data provided by the district did not 
include teachers in charter schools. Analyses of students’ access to effective teachers were limited to teachers who taught math 
and English language arts in grades 3-8, the subjects and grades for which there were annual state student assessments. Because 
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PVAAS scores are calculated for teachers in these subjects in grades 4-8, they are likely to have the most accurate student link- 
ages. PVAAS scores for 2016/17 are excluded from the aggregate findings presented in the main report because the Pennsylvania 
Department of Education changed the way PVAAS scores are calculated that year, and fewer teachers received scores. Because 
the teachers with PVAAS scores in 2016/17 may not be representative of all teachers with PVAAS scores, including those scores 
could bias the results. 

As noted, data were not available for all variables in all years. To maximize analyses of available data while ensuring com- 
parability across findings, the study focused on two periods. Analyses that included teacher characteristics other than teacher 
effectiveness used data from 2010/11 through 2016/17. Analyses that included teacher effectiveness data or school characteristics 
used data from 2014/15 through 2016/17, the most recent years with complete data. 

Complete data on teacher employment was available for 2010/11 through 2016/17, so analyses for research question 3 includ- 
ed the full sample of teachers. For research questions 3a and 3b, however, the sample was limited to teachers with both employ- 
ment data and data on all teacher and school characteristics used as variables. This limited these analyses to teachers employed 
from 2014/15 through 2016/17, because teacher survey data and teacher effectiveness data were available only for these years. 
The characteristics of the sample for research questions 3a and 3b are similar to the characteristics of the larger sample for the 
other research questions, except that the sample for research questions 3a and 3b includes a smaller proportion of teachers with 
four or fewer years of teaching experience. (See tables A8—A11 in appendix A for a comparison of the teacher characteristics for 
the two samples.) 


Methodology. The study analysis methods are briefly described below. Appendix A includes further details. 


Research question 1. To examine whether teacher effectiveness was distributed evenly across the district, the study team used an 
intraclass correlation to measure how similar teachers were within the same school and how similar each school’s average teacher 
was to the average teacher at other schools. 

To determine whether teachers of economically disadvantaged, Black, and Hispanic students were as effective as teachers of 
non-economically disadvantaged and White students, the study reports the effective teaching gap. This districtwide measure is 
based on the difference in the average effectiveness scores and value-added scores of teachers of economically disadvantaged, 
Black, and Hispanic students and the scores of teachers of non—economically disadvantaged and White students. For this research 
question the study used teacher effectiveness scores and PVAAS scores as measures of teacher effectiveness rather than the 
evaluation rating categories because the scores provide a continuous measure of teacher effectiveness. Differences in scores are 
assumed to be meaningful, even for teachers with the same evaluation rating. 


Research question 2. Data on teacher employment in September, October, and April of each year and data on teacher resignations, 
retirements, and terminations in each year were used to calculate the percentages of teachers who left their school and who left 
the district between and during school years. 


Research question 3. The study examined the number of years teachers remained in their school or district before leaving. The 
likelihood that teachers would leave in any given year was calculated without considering differences in turnover by teacher or 
school characteristics. 


Research questions 3a and 3b. To examine whether the likelihood of teachers leaving was related to teacher or school character- 
istics (research question 3a), the study team modeled which teacher and school characteristics were related to teachers leaving 
their school or the district and the timing of when they left while accounting for other characteristics in the data. As noted, 
this sample was limited to teachers employed in the district in school years 2014/15 through 2016/17 because data on all school 
and teacher characteristics were available only for those years. The results in the main report focus on the characteristics that 
have the strongest relationship with teacher turnover and that the district could influence. Turnover rates for groups of teachers 
with differences of at least 5 percentage points are presented in tables in the main report, while other characteristics that have 
a meaningful association with turnover are discussed in the text. To determine whether the relationship between teacher and 
school characteristics and turnover differed between effective teachers and ineffective teachers (research question 3b), the study 
team identified which characteristics were related to turnover for teachers with a rating of failing or needs improvement and 
examined whether that relationship was substantially stronger or weaker for teachers with the two higher evaluation ratings 
(proficient and distinguished). 
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Limitations. Although the data used in this study captured several important teacher and school characteristics, unmeasured 
factors, such as the quality or type of preparation teachers received or teachers’ induction and mentoring experiences, could also 
have influenced teacher turnover. In addition, teacher survey results may not be representative because of low response rates or 
because the survey constructs might not reliably and accurately measure teacher perspectives. School-level response rates on 
the teacher survey ranged from 25 percent to 100 percent, with an average response rate of 64 percent. Response rates were not 
related to schools’ academic performance or student demographics. Because smaller schools had higher response rates, survey 
results may be more representative of teachers in smaller schools than teachers in larger schools. While the teacher survey results 
might have differed if response rates had been higher, the survey results included a large enough fraction of SDP teachers that 
they can provide suggestive information to SDP. 

There were also limitations in measuring teacher effectiveness using SDP’s evaluation data. Teacher effectiveness scores 
include multiple measures, each of which could introduce bias into the analysis of research question 1 and lead to overstating 
inequities in access to effective teachers. Effectiveness scores include a measure of schoolwide performance, both academic and 
nonacademic, that is the same for all teachers in a school. The inclusion of this schoolwide measure could magnify differenc- 
es in average scores between schools and minimize differences within schools. Effectiveness scores also include a measure of 
teacher practice based on principal observations of teachers in the classroom. Research indicates that observation measures 
may be biased, with teachers of lower-achieving students and students of color more likely to receive lower scores than teachers 
of higher-achieving students and White students (Campbell & Ronfeldt, 2018; Gill et al., 2016; Steinberg & Garrett, 2016; White- 
hurst et al., 2014). Such bias could lead to overestimating differences in the quality of teachers of different groups of students. 
Similarly, the PVAAS scores included in the effectiveness scores could overestimate differences in teacher quality because they 
do not include controls for some student characteristics, such as special education status and English learner status, and do not 
account for the effects of a student’s classmates on the student’s achievement (Kane, 2017; McCaffrey & Buzick, 2014; Isenberg 
et al., 2016). PVAAS scores are based on both the size and the precision of the teacher’s estimated value-added score. If teachers 
of lower-achieving students have a less precise value-added score, they are less likely to have very low or very high PVAAS scores 
because of how the scores are calculated, whereas teachers of students who are higher-achieving are more likely to have very 
low or very high PVAAS scores. To address potential bias and limitations in the effectiveness scores and PVAAS scores, the report 
presents results using both measures and discusses similarities and differences in the findings. 

There were also limitations related to the study samples. The sample for research questions 3a and 3b was limited to teachers 
employed during 2014/15 through 2016/17 because data on all teacher and school characteristics required for the analysis were 
available only for those years. This sample included a smaller proportion of new teachers than the district’s teacher workforce as 
a whole, meaning that the findings might not be representative of all teachers in the district. The samples for the analyses using 
PVAAS scores are also not representative of all teachers in the district because only teachers who taught math or English language 
arts in grades 4—8 received PVAAS scores and were thus included in the samples for those analyses. 

Finally, the study does not take into account the influence of the state of the economy on teacher turnover during the years 
studied. Appendix A includes additional information on study limitations. 


Findings 


Effective teachers were distributed unevenly throughout the district, but gaps in access to effective 
teachers differed for different measures of teacher effectiveness 


Both teachers’ effectiveness scores and PVAAS scores on the district’s educator evaluation system differed 
between schools in the district. Teachers of economically disadvantaged, Black, and Hispanic students had lower 
effectiveness scores than teachers of non—economically disadvantaged and White students. But these differenc- 
es were smaller when teacher effectiveness was measured using PVAAS scores alone. 


Teacher effectiveness scores varied between and within schools. Teacher effectiveness scores varied both between 
and within schools, as illustrated in figure 1, which shows the highest, lowest, and average effectiveness score 


for each school serving kindergarten—grade 5 in 2016/17. Average teacher effectiveness scores varied between 
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Figure 1. Average teacher effectiveness scores in the School District of Philadelphia differed between and 
within schools serving kindergarten through grade 5, 2016/17 
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Schools serving grades K-5 


Note: For schools serving elementary school grades in 2016/17, differences in average teacher effectiveness between schools explained 34 percent 
of the variation in teacher effectiveness scores (see table B2 in appendix B). This represents the most recent year of data with a result closest to the 
weighted average across all grades and years (see figure 2). 


Source: Authors’ analysis of data from the School District of Philadelphia on schools serving grades K—5 in 2016/17. 


schools, with scores ranging from 1.5 (just barely rated proficient) to 2.7 (distinguished) on a scale of O-3. There 
was also variation within schools, with about a 1-point difference between the lowest- and highest-scoring teach- 
ers in each school, a difference that was also large enough to put the lowest- and highest-scoring teachers into 
different rating categories. Results across all grade levels and years were consistent with those for 2016/17 and 
indicated that effective teachers, as measured by SDP’s evaluation system, were distributed unevenly between 
schools (see table B2 in appendix B). 


Of the variation in teacher effectiveness scores, about 31 percent was due to differences between schools, and 
69 percent was due to differences among teachers in the same school (figure 2). Had teacher effectiveness scores 
been evenly distributed between schools, each school would have had the same average teacher effectiveness 
score, all variation in teacher effectiveness in the district would be explained by differences among teachers within 
schools, and none of the variation would be explained by differences in average effectiveness between schools. 


Effectiveness scores are calculated using multiple measures, including PVAAS scores and other measures that may 
be influenced by how the evaluation system is implemented in a school, such as differences in teacher obser- 
vation practices by school principals, along with a schoolwide measure of performance that is common across 
all teachers in a school. The inclusion of these last two measures could inflate differences between schools in 
teacher effectiveness scores. Because PVAAS scores are based solely on the academic achievement growth of a 
teacher’s students, differences between schools may be smaller when PVAAS scores alone are examined. Indeed, 
the variation in PVAAS scores due to differences between schools (17 percent) was roughly half the variation 
found for effectiveness scores (31 percent; see figure 2). In addition, there were differences across grades and 
years in the variation in PVAAS scores (see table B2 in appendix B). 


Economically disadvantaged, Black, and Hispanic students had teachers with lower effectiveness scores, on average, 
than did non—economically disadvantaged and White students, but the differences were smaller for PVAAS scores. 
Most students had a teacher rated proficient or distinguished (the two highest of four evaluation ratings in the dis- 
trict’s evaluation system) based on teachers’ effectiveness score. However, economically disadvantaged students 
were slightly less likely (94 percent) than non—economically disadvantaged students (96 percent) to have a teacher 


REL 2020-037 7 


, 


Figure 2. Differences between schools in the School District of Philadelphia were larger for teachers 
effectiveness scores than for teachers’ PVAAS scores, 2014/15—2016/17 
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Note: Figure shows the weighted average across all grades and years of the percentage of variation in average teacher effectiveness explained by differ- 
ences among teachers within schools and by differences between schools. PVAAS score results are the weighted average across 2014/15 and 2015/16 for 
the grades and subjects for which teachers receive PVAAS scores (grades 4—8 and math and English language arts). 


Source: Authors’ analysis of data from the School District of Philadelphia for 2014/15—2016/17. 


rated proficent or distinguished (figure 3). Similarly, Black and Hispanic students were less likely (93 percent and 
94 percent) than White students (98 percent) to have a teacher rated proficient or distinguished. 


To put the magnitude of these differences in context, consider that the difference in average teacher effective- 
ness scores by students’ economic status was similar to or larger than the difference between early career teach- 
ers (teachers in their first four years of teaching in the district) and veteran teachers (teachers with five or more 
years of experience in the district). This difference was typically consistent across grades, subjects, and years (see 
tables B3—B6 in appendix B). Differences in effectiveness scores between White and Black students and between 
White and Hispanic students were larger than the difference between early career teachers and veteran teachers 
for all grades, subjects, and years. 


Analyses using PVAAS scores, which do not include a teacher observation component, showed smaller differenc- 
es by students’ economic status or race/ethnicity than analyses using teacher effectiveness scores. This suggests 
that differences in effectiveness scores might be larger due to the observational component of the effective- 
ness score. The difference in average PVAAS scores by students’ economic status was about half the difference 
between PVAAS scores of early career teachers and veteran teachers in middle school grades and one-third the 
magnitude of differences between early career and veteran teachers in elementary school grades (see tables 
B3-B6 in appendix B). Differences in average PVAAS scores by students’ race/ethnicity were more than half the 
magnitude of differences between early career and veteran teachers in all subjects and years for middle school 
grades and in all years but one in elementary school grades. 


Although differences in average PVAAS scores were smaller than differences in average teacher effectiveness 
scores, economically disadvantaged students were still slightly less likely (92 percent) than non—economically 
disadvantaged students (94 percent) to have a teacher rated proficient or distinguished based on PVAAS scores 
(see figure 3). Black and Hispanic students were also less likely (92 percent and 90 percent) than White students 
(97 percent) to have a teacher rated proficient or distinguished based on PVAAS scores. However, these differenc- 
es could be due partly to the fact that some student characteristics, such as special education status and English 
learner status, and the influence of students’ peers on achievement are not included in calculating PVAAS scores, 
which could result in lower PVAAS scores for teachers of lower-achieving students. 
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Figure 3. Smaller percentages of economically disadvantaged, Black, and Hispanic students than of non— 
economically disadvantage and White students in the School District of Philadelphia were taught by teachers 
rated proficient or distinguished, 2014/15—2016/17 
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Note: Effectiveness score data include students in tested grades and subjects (grades 3-8 and math and English language arts) for school years 2014/15 
through 2016/17. PVAAS score data include students in grades and subjects for which teachers receive PVAAS scores (grades 4—8 and math and English 
language arts) for school years 2014/15 and 2015/16. 


a. The bottom two evaluation rating categories (failing and needs improvement) were combined because of small sample sizes in the failing category. 


Source: Authors’ analysis of data from the School District of Philadelphia for 2014/15-2016/17. 


Differences between schools in average teacher effectiveness scores appear to have driven the differences in 
average teacher effectiveness by student economic status and race/ethnicity. The teachers in schools that eco- 
nomically disadvantaged, Black, and Hispanic students attended had lower scores on average than the teachers in 
schools that non—economically disadvantaged and White students attended. Within schools there was little dif- 
ference in average teacher effectiveness scores by student economic status. In other words, economically disad- 
vantaged students were more likely to attend schools whose teachers had lower effectiveness scores on average, 
but they were not more likely to be assigned teachers with lower scores within those schools. Because there are 
few White students in some schools and high concentrations of White students in other schools, there is little 
room for within-school differences in average teacher effectiveness by student race/ethnicity. 
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On average from 2010/11 through 2016/17, 25 percent of teachers left their school each year 
(17 percent changed schools within the district and 8 percent left the district), with turnover highest in 
teachers’ first five years of teaching 


One-quarter of teachers left their school each year, on average, with most teachers changing schools within 
the district (17 percent) rather than leaving the district (8 percent). The district experienced higher-than-aver- 
age teacher turnover in 2010/11 during a districtwide initiative that converted some schools into charter schools 
(charter schools are not part of the district, so teachers remaining in these schools were recorded as having left 
the district), and in 2012/13, when a budget shortfall led to layoffs and school closings (figure 4). In the years fol- 
lowing these two events, 2014/15 through 2016/17, 21 percent of teachers left their schools each year, on average, 
with 14 percent changing schools within the district and 7 percent leaving the district. 


Across all years from 2010/11 through 2016/17 approximately 3 percent of teachers, on average, left their school 
within the school year, which could create the most disruption for students, while 23 percent left between school 
years (see table B7 in appendix B).' 


The rate at which teachers left both their school and the district from 2010/11 through 2016/17 was highest in the 
first few years of teaching and then stabilized after five years (figure 5). The percentage of teachers leaving the 
district declined between their first and fifth years of teaching and then increased slightly after their fifth year. 
This temporary increase in turnover might reflect the fact that most teachers become eligible for retirement 
benefits in their fifth year. Turnover rose again as teachers approached retirement after about 30 years of teach- 
ing. Five years after beginning teaching in a school in the district, 77 percent of teachers had left their school and 
45 percent of teachers had left the district (figure 6). These percentages change little when years with higher- 
than-average turnover are excluded (not shown). The rates are consistent with those in other urban districts in 


Figure 4. Teacher turnover in the School District of Philadelphia was highest in 2010/11 and 2012/13 but varied 
little in other years, 2010/11-2016/17 
= Total turnover: teachers leaving their == Teachers changing schools = Teachers leaving 
school and leaving the district within the district the district 
Percent 
40 


30 


2010/11 2011/12 2012/13 2013/14 2014/15 2015/16 2016/17 


Source: Authors’ analysis of data from the School District of Philadelphia for 2010/11-2016/17. 


1. Teachers who left between October and April of a school year were defined as leaving within the school year. Teachers who left 
between April and the following October were defined as leaving between school years. The percentage of teachers leaving during 
the school year and the percentage leaving between school years does not sum to the percentage leaving during the full year because 
some teachers may have left their school during the school year and then changed schools again between school years. In these cases 
the teacher is counted both as changing schools during the school year and changing schools between school years but is counted only 
once as changing schools during the full year. 
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Figure 5. The percentage of teachers leaving their school or the School District of Philadelphia was highest 
during the first five years of teaching, 2010/11-2016/17 


Percent leaving school Percent leaving district 
50 50 
40 40 
30 30 
20 20 
10 10 

0 0 

0 5 10 15 20 25 30 0 5 10 15 20 25 30 
Years of teaching in school Years of teaching in the district 


Source: Authors’ analysis of data from the School District of Philadelphia for 2010/11-2016/17. 


Figure 6. The cumulative percentage of teachers leaving their school or the School District of Philadelphia 
increased fastest during the first five years of teaching, 2010/11-2016/17 


Percent leaving school Percent leaving district 
100 : 100 
80 80 
60 60 , 
40 40 
20 20 
0 0 
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Years of teaching in school Years of teaching in the district 


Source: Authors’ analysis of data from the School District of Philadelphia for 2010/11-2016/17. 


the country. For example, research on turnover in 16 urban districts found that 70 percent of new teachers left 
their school within five years and 55 percent left their district (Papay et al., 2017). 


Turnover was highest for teachers of middle school grades, teachers who were frequently absent, 
teachers who identified as Black, and teachers who had previously changed schools 


Even after the relative importance of other characteristics was accounted for in predicting turnover, several char- 


acteristics examined in the study had a substantial influence on teacher turnover (see appendix B for details). The 
findings discussed in this section focus on characteristics that had the largest relationship with teacher turnover, 
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excluding some that the district could not realistically influence (see tables B15 and B16 in appendix B for addition- 
al findings). 


Among teacher characteristics the following had the strongest relationships with teacher turnover: 

© Grade level taught. Middle school and high school teachers had higher turnover rates than elementary school 
teachers (table 1). When multiple characteristics were taken into account, however, high school teachers’ likeli- 
hood of leaving was not substantially different from that of elementary school teachers (see table B15). 

e Teacher attendance. Frequently absent teachers had higher turnover rates than teachers who attended regu- 
larly (see tables 1 and B15). 

¢ Teacher race/ethnicity. Teachers who identified as Black were 30 percent more likely to leave their school than 
teachers who did not identify as Black, after other characteristics were accounted for (see table B15). 

e Teachers’ prior number of school changes. Teachers who had previously changed schools had a higher likeli- 
hood of leaving their school. Teachers with two prior school moves were 21 percent more likely than teachers 
with no prior school moves to change schools again (see table B16). 


Teachers rated failing or needs improvement left their school at a rate more than three times higher 
than teachers rated distinguished 


Turnover also differed substantially and consistently by teachers’ evaluation ratings (see tables B10, B11, and B16 in 
appendix B). Some 40 percent of teachers rated failing or needs improvement based on their effectiveness scores 
left their school compared with 11 percent of teachers rated distinguished (table 2). However, just 12 percent 
of teachers rated failing or needs improvement left the district, meaning that many of these teachers changed 
schools but continued to teach in the district. Among teachers rated proficient, 17 percent left their school while 
just 5 percent left the district each year, and among teachers rated distinguished, 11 percent left their school and 
4 percent left the district each year. 


Table 1. The percentage of teachers leaving their school in the School District of Philadelphia was highest for 
middle school teachers and for frequently absent teachers, 2011/12-2016/17 


Grade taught 

Elementary school grades (kindergarten—5) 21.8 
Middle school grades (6-8) 28.5 
High school grades (9-12) 27.8 
Attendance 

Regular (95 percent attendance or greater) 18.9 
Frequently absent (75-94 percent attendance) 29.0 


Note: Differences between at least two groups within each characteristic are 5 percentage points or more. 


Source: Authors’ analysis of data from the School District of Philadelphia for 2010/11-2016/17. 


2. This excludes findings related to teachers’ retirement eligibility, which is determined by the state; findings for teachers with extended 
absences, which are likely due to extenuating circumstances; and findings for characteristics that would require an unrealistically large 
change to meaningfully influence teacher turnover. 
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Table 2. Teacher turnover was highest for teachers with the lowest evaluation ratings, 2014/15-2016/17 


Percent leaving Percent leaving 
Teacher evaluation rating and score school district 
Failing or needs improvement (O-1.49)? 39.7 12.0 
Proficient (1.5—2.49) 171 4.8 
Distinguished (2.5—3) 11.0 3.5 


Note: Differences between at least two groups within each characteristic are 5 percentage points or more. 
a. The bottom two evaluation rating categories (failing and needs improvement) were combined because of small sample sizes in the failing category. 


Source: Authors’ analysis of data from the School District of Philadelphia for 2010/11-2016/17. 


Teachers were more likely to leave a school that had fewer positive responses on school climate 
survey questions, and the relationship was stronger for teachers rated proficient or distinguished 
than for teachers rated failing or needs improvement 


Although teachers’ responses to survey questions on school culture and climate were generally positive, respons- 
es were less positive at the schools that teachers left than at the schools where teachers stayed. In particular, 
teachers who left were more likely to be teaching at schools in which a lower percentage of teachers responded 
positively to questions about school climate: teachers who left their school taught at schools where 51 percent of 
teachers responded positively to questions about school climate, whereas teachers who stayed taught at schools 
where 57 percent of teachers responded positively (figure 7). A 10 percentage point increase in the schoolwide 
rate of positive responses to questions about school climate—for example, a change from 56 percent of teach- 
ers responding positively (the district average) to 66 percent responding positively—was related to a 15 percent 
decrease in the likelihood of teachers leaving their school (see table B16). 


School climate had a stronger relationship with turnover for teachers rated proficient or distinguished than for 
teachers rated failing or needs improvement. For teachers rated failing or needs improvement, a 10 percentage 
point increase in positive responses to school climate survey questions was related to a 5 percent increase in the 
likelihood of leaving their school (see tables B18 and B19 in appendix B). This would shift a teacher’s likelihood of 
leaving from, for example, 20 percent to 21 percent. However, for teachers rated proficient or distinguished, a 


Figure 7. Teachers were more likely to stay in a school whose school climate teachers rated positively, 
2014/15-2016/17 


@ Schools that teachers left m™ Schools where teachers stayed 


School leadership 
School climate? 
Teacher autonomy 
Teacher respect 


Student behavior 


ie) 20 40 60 80 100 


Percent of positive responses 


a. Difference is greater than 5 percentage points. 


Source: Authors’ analysis of data from the School District of Philadelphia for 2014/15—2016/17. 
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10 percentage point increase in positive responses to school climate survey questions was related to an 18 percent 
decrease in the likelihood of leaving their school (for example, from 20 percent to 16 percent). There were no 
substantial differences by teachers’ evaluation rating in the relationship between turnover and other teacher and 
school characteristics. 


Implications 
Addressing unequal access to effective teachers 


As measured by overall evaluation ratings, teacher effectiveness varied across the district, with teachers in some 
schools more likely to be rated highly than teachers in other schools. This uneven distribution results in econom- 
ically disadvantaged, Black, and Hispanic students being taught by teachers with lower effectiveness scores than 
non-economically disadvantaged and White students. To reduce gaps in access to efffective teachers, SDP might 
want to target its professional development supports for improving teacher effectiveness to teachers of econom- 
ically disadvantaged, Black, and Hispanic students. SDP might also consider how to attract and retain effective 
teachers in schools with higher populations of economically disadvantaged, Black, and Hispanic students. 


However, because the study found that gaps in access to effective teachers were smaller when teacher effec- 
tiveness was measured using PVAAS scores than when using effectiveness scores, SDP might want to investigate 
whether teachers of some groups of students are more likely to receive high ratings on observational measures 
and whether that indicates potential bias. The study’s finding is consistent with research that has found small dif- 
ferences in the effectiveness of teachers of economically disadvantaged students and students of color compared 
with teachers of other students when effectiveness is measured using value-added scores (such as PVAAS scores), 
indicating that some of the gap found when using overall teacher effectiveness scores may be due to bias in the 
observational measures of teacher practice (Isenberg et al., 2016; Mansfield, 2015; Sass et al., 2010; Steele et al., 
2015). That means that if observed gaps in teacher evaluation ratings are driven by bias in measures rather than 
true differences in teacher performance, equalizing access to teachers with high effectiveness scores will not by 
itself address gaps in student achievement. This would be consistent with research indicating that eliminating 
gaps in access to quality teachers is unlikely to substantially reduce gaps in student achievement (Chetty et al., 
2014; Isenberg et al., 2016; Mansfield, 2015). 


Investigating the role of turnover in access to effective teachers 


This study, which explored the relationship between teacher turnover and students’ access to effective teachers, 
did not explore whether vacancies in schools with higher percentages of economically disadvantaged, Black, and 
Hispanic students are being filled by teachers with lower evaluation ratings. SDP might want to consider inves- 
tigating where teachers move within the district and whether where they move differs by teacher evaluation 
rating, to shed light on whether teacher turnover is contributing to inequities in access to effective teachers. 
However, some research suggests that teachers moving to another school or out of the district does not contrib- 
ute to meaningful gaps in access to effective teachers (Isenberg, et al., 2016; Sass, et al., 2010; Steele, et al., 2015). 


Attention to factors that influence teacher turnover 


Consideration of the characteristics identified in this study as being related to teacher turnover, particular- 
ly among teachers rated proficient or distinguished, could help refine the district’s ongoing efforts to improve 
teacher retention by providing targeted professional development opportunities and analyzing administrative 
data to improve school climate. Based on findings in this study, the district could develop an early warning system 
to identify teachers likely to leave, paying particular attention to middle school teachers, teachers who are fre- 
quently absent, teachers who have previously changed schools, and teachers who receive low evaluation ratings. 
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District leaders could then offer additional support for those who need it, such as providing extra coaching and 
help from school leaders and colleagues, reducing teachers’ nonteaching responsibilities, and taking steps to 
address teachers’ stress and anxiety (Knoster, 2016; Pallas & Buckley, 2012). 


The study also found that teachers who identify as Black were more likely to leave than other teachers, after 
other teacher and school characteristics were accounted for, which could indicate that these teachers experience 
more challenging working conditions than other teachers (Dixon et al., 2019; Ingersoll et al., 2018). A body of 
research is emerging on the benefits to students of having a teacher who shares their racial/ethnic identity (Dee, 
2004; Gershenson et al., 2016; Gershenson et al., 2017). Given the high proportion of students of color in SDP, 
the district might explore underlying reasons why Black teachers are more likely to leave, such as through focus 
groups or teacher surveys, which could help identify strategies that emphasize the retention of teachers of color. 


The finding that school climate influenced teacher turnover suggests that improving school climate is another 
way to strengthen teacher retention, particularly for proficient and distinguished teachers. Prior research has 
also found that working conditions are an important factor in teachers’ decisions to leave their school (Loeb 
et al., 2005; Pallas & Buckley, 2012). Findings from the current study indicate that teachers with higher evaluation 
ratings may be more willing or able to leave schools with a poor school climate and more likely to stay in a school 
with a positive school climate, making this an important factor in school-level efforts to retain effective teachers. 
The district could use teacher responses to its annual surveys to identify schools with a below-average school 
climate and work with school leaders and staff to identify policies or practices that might establish a more posi- 
tive school culture. 


As the district considers strategies to improve teacher retention, it could also consider the type of turnover it 
seeks to address. For example, replacing a teacher who is terminated for poor performance or other harmful 
behavior with another teacher might improve student achievement or school climate, and replacing an experi- 
enced teacher who retires with a new, less experienced teacher may result in cost savings for the district. 


Using existing research on teacher turnover and investing in additional research 


Since the turnover rate in the district is consistent with that in other urban districts, SDP might want to consider 
strategies used by other districts to retain effective teachers in struggling schools. For example, some districts 
have experimented with incentives for teachers to stay at their school, such as selective retention bonuses for 
effective teachers in targeted schools (Swain et al., 2019). Other districts have experimented with financial incen- 
tives that transfer effective teachers from high-performing schools to lower-performing schools (Glazerman et al., 
2014). Some states and districts have created career advancement pathways to encourage effective teachers to 
stay in their classrooms while taking on leadership roles within their school (Sutcher et al., 2016; Workman & 
Wixom, 2016). 


The district might also consider ways to use the administrative data employed in this study and other available 
data to identify and test potential retention strategies. Incorporating newly collected data on teachers’ preservice 
experience, such as teachers’ certification pathway, could help identify prospective teachers who are more likely 
to stay in their school or in the district. Finally, the district might benefit from exploring the relationship between 
teacher turnover and other characteristics not examined in this study. For example, turnover could be related to 
principal quality, as measured by principals’ evaluation ratings, or to student behavior, as measured by student 
disciplinary records. The district could use the data included in this study and other data to develop a model for 
identifying teachers with the highest risk for turnover and connecting them with additional supports. 
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